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computing 


Our Image Transmission Challenge 
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> Hard Environments: 
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> Fading/Signal Loss 
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> Low compute power 


> Limited time windows 
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> e.g., sSDV 


SSDV 


> Encode image as JPEG 


> Transmit in packets of JPEG minimum coded 
Units (MCU) 


> Optional forward error correction (FEC) 
> Excellent results! 
> Two issues: 


> Lost packets = blank parts of image 


> JPEG encoding + FEC hard in low-memory 
devices 


men) Examples 
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Only send 10 packets 


Loni send GO 


Only send 100 packets 


Only send 300 packets 


PCS| Demonstration 
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Not connected > TX Pause 


r TX Continue 
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Host: 


| Connect 


Connected to localhost:8001 


> hitos://madgitrnswa.github.io/PCSI/ 


How does PCSI| work¢ 


FOR MORE INFO ON “COMPRESSED SENSING IMAGING,” CHECK OUT: 


HTTP://WWW.PYRUNNER.COM/WEBLOG/201 6/05/26/COMPRESSED- 
SENSING-PYTHON/ 


Compressed Sensing in Biology 


> How do our eyes work? 
> 120 million rods (greyscale pixels), 6 million cones (color pixels) 


> Humans Perceive ~24 bit color (16mil colors) and ~10bit grey (1000 
levels) 


> Humans perceive ~100 frames per second 
> What is the needed bandwidth¢ 
> (120million*24 bits + é6million*10)*100 = 294 Gb (gigabits) per second 


> Optic nerve can carry less than 10 Mb (megabit) per second (Koch, et. 
al., “How Much the Eye Tells the Brain” Current Biology, 2006) 


> IT’S 30,000 times too slow!!! How can it be??!? 


Compressed Sensing in Biology 


Compressed 


Yr Sensing a 


> Our eye doesn’t have to transfer 
all the imaging data, just 
enough for our brain to 
reconstruct what Is there (or 
perceives Is there} 


> Some amount of neural network 
processing: pattern recognition 
(sends “shapes” not “pixels”), 
only transmit changes, etc. 


> Can we use similar tricks To 
transmit imaging data? 


> Just transmit pixels on left and 
reconstruct image on right 


Why this matters: 


> If we have imaging data in one location, and 
we want to broadcast it, you can: 


> Send it “analog” (e.g., SSTV from the ISS) 
> Send it digitally (in packets) 


> need two way communication to handle lost 
packets (request resend) 


>» Compression need memory, power at transmitting 
end 


> If transmitting without acknowledgements, data is 
lost — corrupting or losing the image 


> What about digital packets that can be lost¢ 


> Do we even need to send everything? 


Do we even need fo transmit all 
the data¢ 


> If your image is “sparse,” you don't need to send every pixel! 


> See paper for details and references 
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Random Pixels in the Packet 
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Need to send random pixels each packet — but 
needs to be deterministic. 


Pseudorandom Number Generators are deterministic 
and sort-of-random! 


Each packet contains data from randomly selected 
pixels and includes the “packet id number” in the 
header. From the packet id number, the receiver 
can deterministically figure out which pixels are 
oresent in the packe?!! 


Another Trick: 
Chroma Compression 


> Human eye: 120 million rods (greyscale pixels), 6 million cones 
(color pixels) 


> Do we need to send high resolution color? 

>» JPEG uses chroma compression (up fo 4x) 

> Chroma Compression in PCSI: 

Represent image as YCbCr instead of RGB 
Send Y data (e.g., B&W) for all pixels 

Send only a fraction of Cb and Cr (color) pixels 
Reconstruct each channel separately. 
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PCSI Chroma Compression of 20 works well! (only send 5% of the 
pixels as color, the rest as black and white!) 


> Optional down size from 24 bit color to lower (e.g., 12 bit) 


hitos://en.wikipedia.org/wiki/Y CoCr 


Packet Framing 


> Any framing works! All that matters in the payload format (see paper 
for soec) 


> Currently implemented: AX.25 
> Existing KISS hardware and software 


> Compatible with APRS hardware/software — but not a good idea to 
soam APRS channels 


> Optional digipeating 
> Future framing: SSDV-style Framing 


> Less overhead, better for higher BER environments 


PCSI Transmit Steps 


> Use a PRNG to select random pixels from the image 
> Chroma compression: select pixels fo be YCbOCr and others just Y 
> Convert to RGB to YCbCr. Downscale bit depth 


>» Create Pseudorandom Datagram Payload (PDP) consisting of 
header w/ image info and pixels 


> Each PDP contains all the information you need to fully reconstruct an 
image. Each additional PDP received increases image quality. 


> Frame It up! 
> Send it off to your TNC to handle flags, bit stuffing, CRC, etc. 


PCSI Receive Steps 


> Extract header and pixel info 


> You now know image size, bit depth, etc. And you have the locations 
and values of the full color (YCbCr) and B&W (Y) pixel channels 


» Use a compressed sensing algorithm to make a best-guess as to 
what each channel originally looked like, individually. 


>» We used: OWL-QN variant of L-BFGS (see paper) to reconstruct a sparse 
DCT spectrum. This is the only computationally intense step of the 
Orocess. 


> Combine reconstructed channels and convert back to RGB 


> Display the image! 


Future of PCSI 


Mawilin 
>» Arduino transmit client 


> Centralized packet Upload i" 
system (similar to SSDV’s) — 
> APRS Frequency Objects 
+ protocols 


> Cell phone apps! 


> Rob Riggs (WX9O) at 
Mobolinkd has been helping 
testing, we’re starting a 
partnership developing PCSI 
phone apps 


APRS» Made Simple 


® APRS is a registered trademark of Bob Bruninga WB4APR 


Thank you, any questions¢ 
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Rescan Ports | TX Start/Restart | 
Connect = TX Status: Running 


Not connected > TX Pause 


r TX Continue 


KISS TCP Config | Load Image | 


Host: 


| Connect 


Connected to localhost:8001 


> hitos://madgitrnswa.github.io/PCSI/ 


Math of compressed sensing (more 
detailed) 


> |If| have an image and only transmit 
a random subset of pixels, I’m doing 
this math: 


Ax =b 


> x’ is a single vector that is a list of the 
value of each pixel from the original 
image 


> “b"” isa single vector that ts a list of 
the pixel values received at the end 
of the transmission 


> “A” is the “measurement” matrix 
(example next slide) 


Math of compressed sensing (more it 
detailed} 


> Let’s say we have a 3x3 image (9 pixels) 
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If we only transmit some pixels To 
OD... 


> Let’s only transmit a few pixels (Counting from the top, starting with OQ): 
p> First pixel 5 (x[5]), then | eo then 7 (x[7]) to b 


“b” vector is what we receive! 
0 00 0 0 de GmOmeG a 
Jorco0000q 
v1 0000009 


What if we are b¢ What do we 
Know? 
> Now let’s just pretend we're b, and we see the pixels that were 


transmitted. Can we figure out what the original image is? 
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> Too many unknowns!! 


>» Need more constraints! 


What if we are b¢ What else do we 
knowe¢ 


> Now let’s just pretend we're b, and we see the pixels 
that were transmitted. Can we figure out what the 
original image Is¢ 


> 
> 


Too many unknowns!! 


But we do know that, for the most part, if | take a Fourier 
transform of most signals (and images), most of the 
frequencies are zero. 


If | know that most of values of the frequency space Is 
zero, then | might have enough measuremenis to solve 
my problem! 


NOTE: instead of Fourier transform, people use a “Discrete 

Cosine Transform” (DCT) which is very similar. Maybe we'll 

get to details later, for now — it’s just like an FFT but it is only 
real (no phase) 


Ax 


What if we are bo¢ What else do we 
knowe¢ 


> So let’s transform our equation in to the frequency domain. “x” is the list of 
pixel values, “X” is the DCT spatial frequencies 


dct(x) = X 
idct(X) =x 


= 


Ax = b ARE spr. ine 
A idct(X) — i 9 test image DCT spectrum 


>» We know A and b, so we want fo find “X” in: A idct(X) — b = 0 


> We want to solve it using the “X” that has the fewest NONZERO elements 
as possible (most of X are zeros) 


How fo find the right “xX” 


> “Minimizing number of non-zeros” is hard for a computer to do 


> It’s called an “LO norm” 


>» However, people found that if you instead minimize the “L1 norm,” 
you pretty much get the same answer. 


> Minimizing the L1 norm means just minimizing the sum of the abs of each 
element: »|X;| 


> OK, so in the end, what we want: 


> Find (“guess”) the “frequency components” X that minimizes the error 
between what we received and what we guess AND minimizes the “L1 


“a L1norm = ||X||1 = > Ie 
error = » |A idct(X) —b |? 


How we “guess” the right X! 


Minimize X = »1A idct(X) — b |* + CY 1Xe 


> Lots of math, but the above is easy! Programs already written to do that problem! 
i/Limited-memory BFGS#OWL-QN 


> hito://www.poyrunner.com/weblog/2016/05/26/compressed-sensing-python 
>» Python: hitos://bitbucket.ora/rtaylor/pylotas 
> C: hito://www.chokkan.orag/sottware /liblotags 


>» What do you need fo do it? 


> “b”-—a vector of the pixels we received and we know the value 


> “C”-a “weighting” coefficient for how much we want to “prioritize” finding a solution 
has the least non-zero values in X. C Is typically between 3 and 5. 


> The program will just find us “X” that fits the above equation! 


Our example: 


> Find the DCT coefficients (X,) that minimize below: 
Z 


0 0 0 0 0 GiOmen ome. (X02 57 
> YI10 10 00 0 0 O Ofidct cee: —112]] — DIX, | 
00000001 0 Xoo) X22 I atten 1 


> We set this up, and let the computer guess X for us. The computer then asks us to 
find what is the value of the function above, we tell it, and | makes another guess. 


> We also can tell it the derivative to make if faster. 


